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We study a mixture of fermionic and bosonic cold atoms on a two-dimensional optical lattice, 
where the fermions are prepared in two hyperfine (isospin) states and the bosons have Bose-Einstein 
condensed (BEC). The coupling between the fermionic atoms and the bosonic fluctuations of the 
BEC has similarities with the electron-phonon coupling in crystals. We study the phase diagram 
for this system at fixed fermion density of one per site (half- filling). We find that tuning of the 
lattice parameters and interaction strengths (for fermion-fermion, fermion-boson and boson-boson 
interactions) drives the system to undergo antiferromagnetic ordering, s-wave and d-wave pairing 
superconductivity or a charge density wave phase. We use functional renormalization group analysis 
where retardation effects are fully taken into account by keeping the frequency dependence of the 
interaction vertices and self-energies. We calculate response functions and also provide estimates of 
the energy gap associated with the dominant order, and how it depends on different parameters of 
the problem. 
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Recent advances in manipulating ultracold atoms in 
optical lattices have produced remarkable results span- 
ning a variety of modern condensed matter physics phe- 
nomena such as the realization of the Mott-insulator 
transition 1], fermionic superfluidity the realization 
of the Tonks- Girardeau gaslsl, and the Berezinskii- 
Kosterlitz-Thouless transition [4|, just to name a few in- 
dicative ones. The experimental control over a wide 
range of parameters associated with these type of sys- 
tems renders them a favorite physics playground both 
for experimentalists and theorists alike. In particular, 
a boson-fermion mixture (BFM) system can provide a 
fascinating Jesting ground for a large section of theoreti- 
cal physics 15] furthering our understanding of condensed 
matter phenomena. 

Previous work has explored part of the vast and rich 
phase space associated with this two-species system by 
employing mean-field or variational type of approaches 
and allowing for fluctuating occupation numbers for both 
fermions and bosons[6,, 7, 8., 9, lOj. As expected, these ap- 
proaches provide a good general picture of the rich phase- 
space but are unable to capture the delicate interplay 
and competition of correlation effects in a quantitative 
manner which can be a useful reference for experimen- 
talists. A more recent work, based on our methodology of 
analysis [nj, was the first approach to study this type of 
system beyond mean-field limitations but (as in all pre- 
vious studies) was applied only for the regime vp ^ Vg, 
where retardation effects between the fermionic dynam- 
ics dictated by the Fermi velocity vp and the bosonic 
dynamics dictated by the sound velocity Vs are not im- 
portant. It is very important to investigate the physics 
beyond this regime since > is experimentally ac- 
cessible and in fact, retardation can play a crucial role 
in the interplay of orders. The above considerations have 
prompted us to focus our attention on this type of system 
and complement on the physical understanding of it by 



applying our theoretical apparatus originally developed 
for the electron-phonon type of system [lj|. 

The framework under which we work is based on 
functional renormalization group (fRG) analysis for two- 
dimensional fermions 13] in the presence of pho nons when 
full retardation is taken into consider ationyj. This ap- 
proach has the important advantage of including all com- 
peting fermionic processes under a given (one-loop) ac- 
curacy approximation in an unbiased way, so that all 
possible orders are studied on an equal basis, contrary to 
mean field theory for example which always presupposes 
a manifested order. The only necessary ingredient that 
is required for this study to work is the bosonic system 
to be in the Bose-Einstein condensation phase (BEC) 
which leads to an electron-phonon type of fermion-boson 
interaction (linear to first order in the bosonic field) [lH . 
This allows us to integrate the bosons out and build an 
effective fermionic theory, and consequently apply the 
fRG analysis to investigate all possible orders of inter- 
est the fermions can undergo, while taking retardation 
effects fully into account. What is interesting to realize 
is that our study can be generalized for any type of a 
two-component itinerant fermionic system on a square 
lattice in the presence of a gapless collective mode that 
renormalizes attractively the fermionic interaction. All 
of our results will remain valid within that context. 

The standard model used to study itinerant fermions 
and bosons on a lattice is the Hubbard model. For a 
typical BFM system consisting of neutral atoms (*°K 
for fermions and ^^Rb for bosons) that interact through 
short-range van der Waals forces a good approximation 
consists only of hardcore (onsite) intra-species (fermion- 
fermion [///, boson-boson Ubb) repulsive interactions and 
inter-species (fermion-boson Ufb{q)) repulsive or attrac- 
tive interactions. Additionally, the presence of a two- 
component isospin quantum number among fermions re- 
laxes Pauli's exclusion principle limitations for s-wave 



2 



type of scattering events (lowest in energy) and opens up 
a large and rich phase space previously inaccessible for 
"spinless" fermionic atoms. This type of two-component 
fermionic system in the presence of a collective mode 
can effectively map and simulate a large variety of corre- 
lated electronic systems in condensed matter physics [HI . 
The role of isospin in this case is played by different 
total angular momentum projection states for the *'^K 
atoms, which can additionally be exploited for Feshbach 
resonance type of tuning of the inter-species scattering 
strength C//{,[15[. 

The Hamiltonian for this system of itinerant fermions 
and bosons on a two-dimensional square lattice has the 
form 

k,a q 

k,k' .q 

+ ^ E UMfi^^Ju,.Bl_^B^, 

k,k\q,(T 

T7 E ^fffk+q,pfk,Tfl'-q,lfk',l, (1) 
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where fk,cr,Bq are the fermion, boson operators respec- 
tively, £,1.'^ = —2tf^b{coskx + cosky) — jif^b are the corre- 
sponding dispersion relations dictated by the square lat- 
tice symmetry, with tf i, being the overlap integrals and 
fif^h the chemical potentials for the fermions and bosons 
respectively. 

For the purpose of this study we will focus our at- 
tention at half-filling for the fermions (/Lt/ =0) where 
the pairing-related phases (s,d-wave) superconductivity 
(sSC, dSC) can compete with the nesting-related phases 
like charge density wave (CDW) and antiferromagnetism 
(SDW), and a rich landscape of phase space becomes 
available for the fermionic system to explore [l^. This 
amounts to fermion numbers in the range lO'^-lO^ for 
typical optical lattice parameters a — 400nm on 2D op- 
tical harmonic traps of frequency w = 27r x 30Hz, which 
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is well within experimental capabilities. Additionally, at 
half-filling besides the presence of nesting on the Fermi 
surface there are van Hove singularities as well that can 
enhance the interplay and competition among all or- 
ders. Our assumption of BEG for the bosons allows us 
to expand the bosonic operators in Eq. ([1]) according to 
Bq = \/NoSqfi + Bq, whcrc the creation or destruc- 
tion (Bq) of quasiparticles follows from fluctuations (-Bo) 
of the macroscopic number Nq of condensate atoms. Con- 
sequently we can perform the usual Bogoliubov transfor- 
mation and linearize the inter-species interaction intro- 
ducing the gapless collective mode 



where €„ 



eq{eq + 2nbUbb), (2) 
is the free-boson dispersion rela- 



tion (chemical potential exactly at the non-interacting 
ground state), and rii, ~ Nq/V is the EEC density which 
for all practical purposes can be considered as the bosonic 
filling factor on the lattice. 

This is the fixed point of our theory as far as the sta- 
bility of the bosonic system in the presence of fermions is 
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concerned. It has been experimentally shown |1( 
that the BEC can be stable in the presence of fermionic 
atoms provided the two-species numbers do not exceed 
10^ atoms in the optical trap in the absence of an optical 
lattice. The presence of an optical lattice will stabilize 
the system against phase separation or collapse to even 
higher densities, since both processes rely on the inter- 
species interaction Ufb dominating over the kinetic en- 
ergy scales associated with the weakly confined atoms, 
and three-body recombination processes becoming rel- 
evant as well. The lattice will introduce experimentally 
tunable kinetic energy scales associated with the hopping 
of atoms from lattice point to lattice point (t/.fj) compet- 
ing with Ufb and consequently stabilizing the system. 

Dynamical fluctuations of the BFM around this fixed 
point can be modeled in the Matsubara representation. 
After the bosonic fields associated with fiuctuations of 



the BEC are integrated out[ll|,|l^ we arrive at the fully 
retarded effective fermionic interaction given by 



Uff{ki,k2,kz) = Uff - 



bb 



^ki-ks 



1 + 4^2 - 2C2(cos(fci 



k3)x 
I 



cos (fci - k3)y) (Wl - W3)2 -I- Ujl^_k^ 



(3) 



where ki = {iujn^,ki)^ a;„. are the fermionic Matsubara 
frequencies, and ^ = yJtb/2nbUbb is the healing length 
of the BEC. As we see, fluctuations of the BEC renor- 
malize Uff with an attractive and frequency-dependent 
part which is not affected by the attractive or repulsive 



nature of the inter-species interaction U fb 8] . The im- 
portance of retardation is determined by Eq. ([2]) which 
in the longwavelength limit defines the acoustic veloc- 
ity Vs = \/2tbUbbnb- For fermionic velocities (vp — 2tf) 
much smaller than one recovers the mean-field limit 
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FIG. 1: (Color online) Phase diagram for Ujj — OAtf, Utt ~ 
0.8tf, and nt = 2.5. Blue circles indicate sSC, red rhombuses 
indicate dSC, magenta squares SDW, and green stars CDW 
type of ordering. Dashed lines are guides to the eye. 



FIG. 2: (Color online) Phase diagram for (/// — OAtf, 
Ubb = 0.8tf, and rib = 2.5 in the limit vf <^ Va where retar- 
dation is not important. In this case, the whole CDW phase 
disappears. Symbols are same as in Fig. fTJ and dashed line 
is guide to the eye. 



for Uff 8] . On the other hand, retardation dominates 
for Vs < vp and the competition of phases is enhanced. 
This regime can be achieved experimentally since there 
are two tunable parameters (tf,, Ubb) that can be adjusted 
in situ for a given system configuration (rif,). The heal- 
ing length ^ plays an important role in the competition of 
phases since it defines the length-scale over which bosonic 
correlations are present [lo| . If these correlations, which 
enter the attractive part of the fermionic interaction, are 
short-ranged f < 1 (in units of the lattice constant) this 
will favor CDW (at half-filling) competing with s-wave 
type of pairing. If on the contrary ^ > 1, the lattice sym- 
metry becomes visible and exotic pairing phases enter 
the competition 1^, 11 1 along with SDW (at half- filling) . 
This is reflected throughout our results as well as we dis- 
cuss below. 

Our fRG analysis for the fermionic system was ap- 
plied at zero temperature for the electron-phonon sys- 
tem at half- filling [IJ]. It involves the self-consistent in- 
tegration of fast-modes, determined by the energy cut- 
off A(^) = 4t/e~^ {£ is the RG-stcp in the integra- 
tion process) which renormalizes the fermionic interac- 
tion until a divergence occurs as the limit — + oo is 
approached. At half-filling the presence of van Hove 
singularities on the Fermi surface allows us to focus 
our attention only on fc-space processes around the sin- 
gular points Qi — (tt, 0) + k and Q2 — (0, tt) + 
k with |fe| <^ TT, called the two-patch approxima- 
tion. As a result, a general fermion-fermion coupling 
C///(fci, /c2, ^3) reduces in momentum-space to four non- 
redundant couplings associated with the scattering pro- 
cesses of gi = Uff{Qi,Q2,Q2), 92 = Uff{Qi,Qi,Qi), 
93 = Uff{Qi,Qi,Q2), 34 = Uff{Qi,Q2,Qi), where 
we have suppressed the frequency dependence for clar- 
ity. The different orders of interest are defined in terms 



of these couplings according to[l£ 



W(sd)SC = -2(52 ±53), 
USDW = 2(54 + 93), 

ucBW = -2(2.gi + .93 - .94), 



(4) 
(5) 
(6) 



where the signs are chosen so that all orders diverge 
to positive values when they strongly renormalize. In 
this two-patch approximation the effective interaction of 
Eq. ^ reduces to the pairs of 



gi^3{uJl,LU2,UJ3)^Uff 



92A{uJl,l^2,i^3)^Uff 



Ufb 



8ti/e 



Ubb (^^_^3)2+^(l + 8^2) 



u 



Ubb 



,(7) 
(8) 



which are inherently anisotropic {Si^-^^i^^ is the Kronecker 
delta). This inherent anisotropy can enhance d-wave 
pairing for example in the regime where 92 < 0, 93 > 
with I (73 1 > \g2\ as Eq. ^ shows. Finally, in order to 
account for all frequency channels correctly one needs to 
evaluate the susceptibilities of the different orders and 
identify the most divergent one 14 \ . 

We have performed our fRG analysis and have inves- 
tigated various parameter ranges associated with the ex- 
perimentally tunable set of {[///, J7/b, J7bb, t;,, t/, n;,}. As 
is reasonably expected, for Uj^^/UbbUff < 1 the effective 
interaction of Eq. ([3]) remains repulsive and antiferromag- 
netic ordering dominates throughout the range of param- 
eter space. Interesting physics arises when [/|^/J7f,bC/// > 
1. In Fig. IT]) we present the phase diagram our fRG anal- 
ysis produces for Uff = OAtf, Ubb ~ O.Stf and Ub = 2.5 
and a range of Uj^/UbbUff > 1, tb/tf values. We notice 
that even at half-filling a large region of the phase dia- 
gram is taken over by pairing related phases. This is the 
essential new result of this work and the difference with 
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FIG. 3: (Color online) Evolution of the gap along tb/tj — 
0.6 of Fig. ^ with identical symbol scheme for the different 
orders. The blue line fitting was according to A = O.OfS + 
3.326 exp(-(3.10i + A)/A). 



previous studies, where mean-field[lOj and vp << Vs 
£RG ll| focused on different dopings for the fermionic 
system. Both were unable to capture at half-fiUing the 
rich interplay of orders and their strong dependence on 
tiy. As tb is strengthened, the bosonic correlation length ^ 
(which is independent of U fb) becomes larger allowing for 
boson-mediated attraction among the fermions to reach 
over many lattice sites increasing the Cooper pair sizes 
and allowing fermions to avoid on-site pairing and the 
associated energy cost of C///. For small Ufb values this 
mechanism is detrimental for CDW and only when the 
bosonic correlation length ^ < 1/2 can CDW win over 
s-wave pairing. For larger values of U fb charge order can 
survive over sSC in a region 1/2<^<1. If the range 
of the attractive interaction exceeds the lattice constant 
value then pairing across different sites becomes energet- 
ically favorable and CDW becomes subleading. 

At this point we should mention that the ^ ~ 1/2 
condition for CDW to appear indicates that the BEC 
phase has weakened as a whole and long range order has 
been lost. In free space, the consequence of this is that 
our linear approximation of coupling fermions to only the 
bosonic density fluctuations might not be adequate, there 
will be lots of quasiparticle excitations. Nevertheless, we 
believe that even if those processes are taken into account 
the strong influence of the lattice and the presence of 
nesting on the Fermi surface will sustain the CDW order 
to be experimentally observed. 

As we mentioned in the introduction, all previous 
works on this type of system have ignored retardation ef- 
fects. This is an a priori assumption based on comparing 
the free-space masses of '^'^K and ^^Rb[l3l. When the lat- 
tice is turned on, one should be comparing band-masses 
related to t/, tb which can be tuned to any range and con- 
sequently vp^Vs can be independently set experimentally. 
In Fig. ^ we show the same phase diagram as in Fig. ^ 
without retardation taken into consideration, which ef- 



fectively amounts to setting uji = 003 for all frequency 
channels in Eqs. (I7][8]). We have recovered the results at 
half- filling of the previous fRG study [lH. Antiferromag- 
netism is expected to take over at the U^^/UbbUf f < 1 
region. As it can be clearly seen by contrasting the two 
phase diagrams, retardation is very important at tuning 
the fermionic phases. 

Our analysis has the additional advantage of qualita- 
tively producing the energy gap associated with the dom- 
inant orders, A = 4t/e~^'=, just by following the critical 
RG-step (ic) at which they diverge. We can define 



A 



U%2- 



2Ubb 1 + 8^2 ' 



(9) 



as the average anisotropic coupling strength of the 
fermion-boson interaction (easily derived from Eqs. ([7][8])) 
and investigate the functional form of A (A). In Fig. ([3|) 
we show the evolution of the gap along the tb/tf = 0.6 
direction in Fig. ([T]), where the fermionic system passes 
through all four phases. The functional form we use 
for the fitting is A = di + d2cxp(— (^3 + A)/A) with 
di = 0.015, d2 = 3.326, rfg = 3.101. 

In conclusion we have applied functional renormaliza- 
tion group method with full retardation for a boson- 
fermion mixture system on an optical lattice when the 
fermions are at half-filling and the bosons have Bose- 
Einstein condensed. We find a rich phase diagram pro- 
viding quantitative estimates for the experimental obser- 
vation of the competing orders. Additionally, we have 
produced a functional dependence of the energy gap of 
the dominant orders with the average boson-fermion cou- 
pling strength. 
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